本文介绍了无监督的联合学习框架FEDX。我们的模型从分散和异质的局部数据中学习无偏的表示。它采用对比度学习作为核心组件的双面知识蒸馏，使联合系统可以在不要求客户共享任何数据功能的情况下运行。此外，它的适应性体系结构可以用作联合设置中现有无监督算法的附加模块。实验表明，我们的模型可显着提高五种无监督算法的性能（1.58--5.52pp）。

体育运动员的转会费已成为天文学。这是因为将具有巨大未来价值的球员带给俱乐部对于他们的生存至关重要。我们介绍了一个案例研究，该案例研究基于FIFA数据分析，影响世界顶级足球运动员的转移费用。为了预测每个玩家的市场价值，我们通过使用树结构化的Parzen估计量（TPE）算法优化其超参数来提出改进的LightGBM模型。我们通过Shapley添加说明（SHAP）算法确定突出特征。已提出的方法已与基线回归模型（例如线性回归，拉索，弹性净，内核脊回归）和没有超参数优化的梯度增强模型进行了比较。与回归基线模型，GBDT和LightGBM模型相比，优化的LightGBM模型平均表现出的出色精度约为3.8、1.4和1.8倍。我们的模型在确定未来招募足球俱乐部应考虑的属性方面提供了解释性。

分配和验证国际公认的商品代码（HS编码）的任务是贸易货物的是海关办公室的关键职能之一。这一决定对于进口商和出口商至关重要，因为它决定了关税率。但是，类似于法官作出的法院决定，即使对于经验丰富的海关官员，任务也可能是非琐碎的。目前的论文提出了一个深入的学习模式，以协助这一看似挑战HS代码分类。与韩国海关服务一起，我们建立了基于科电的决策模型，该决策模型建议了HS代码的最有可能的标题和副标题（即，前四位和六位数）。在129,084件之前的情况下评估显示，我们模型的前3个建议在分类265个副标题方面的准确性为95.5％。这个有希望的结果意味着算法可以通过协助HS代码分类任务来减少海关官员所采取的时间和精力。

异常检测旨在识别来自正常数据分布的异常情况。该领域已经取得了许多进展，包括创新使用无监督的对比学习。然而，现有方法通常假设清洁训练数据，并且当数据包含未知异常时受限。本文介绍了一种新型半监督异常检测方法，统一了与无监督的对比学习的能源的模型的概念。 ELSA通过基于新能量函数的精心设计的微调步骤灌输对任何数据污染的鲁棒性，这些步骤迫使正常数据分为原型的类别。多种污染方案的实验表明，所提出的模型实现了SOTA性能。广泛的分析还验证了每个组件在所提出的模型中的贡献。除了实验之外，我们还提供了一种理论解释，对何对象学习独自无法检测到数据污染下的异常。

根据其在高风险方案的部署，调节人工智能（AI）已成为必要的情况。本文探讨了向AI和机器人扩展法人和机器人的提议，尚未通过公众的镜头检查。我们提出了两项研究（n = 3,559），以获得人们对电子法人的看法Vis-\'a-Vis现有责任模型。我们的研究表明，即使这些实体没有承认任何精神状态，人们即使这些实体也无法承认自动化代理人的愿望。此外，人们并不相信自动化代理人的惩罚将达到威慑性和报应，并不愿意授予他们的法律惩罚前提，即物理独立和资产。共同努力，这些调查结果表明，惩罚自动化剂的愿望与其感知不切实际之间的冲突。我们通过讨论未来的设计和法律决策如何影响公众对自动化代理的不法行为的反应方式的结论。

Harmonic functions are abundant in nature, appearing in limiting cases of Maxwell's, Navier-Stokes equations, the heat and the wave equation. Consequently, there are many applications of harmonic functions, spanning applications from industrial process optimisation to robotic path planning and the calculation of first exit times of random walks. Despite their ubiquity and relevance, there have been few attempts to develop effective means of representing harmonic functions in the context of machine learning architectures, either in machine learning on classical computers, or in the nascent field of quantum machine learning. Architectures which impose or encourage an inductive bias towards harmonic functions would facilitate data-driven modelling and the solution of inverse problems in a range of applications. For classical neural networks, it has already been established how leveraging inductive biases can in general lead to improved performance of learning algorithms. The introduction of such inductive biases within a quantum machine learning setting is instead still in its nascent stages. In this work, we derive exactly-harmonic (conventional- and quantum-) neural networks in two dimensions for simply-connected domains by leveraging the characteristics of holomorphic complex functions. We then demonstrate how these can be approximately extended to multiply-connected two-dimensional domains using techniques inspired by domain decomposition in physics-informed neural networks. We further provide architectures and training protocols to effectively impose approximately harmonic constraints in three dimensions and higher, and as a corollary we report divergence-free network architectures in arbitrary dimensions. Our approaches are demonstrated with applications to heat transfer, electrostatics and robot navigation, with comparisons to physics-informed neural networks included.

We propose Universal Document Processing (UDOP), a foundation Document AI model which unifies text, image, and layout modalities together with varied task formats, including document understanding and generation. UDOP leverages the spatial correlation between textual content and document image to model image, text, and layout modalities with one uniform representation. With a novel Vision-Text-Layout Transformer, UDOP unifies pretraining and multi-domain downstream tasks into a prompt-based sequence generation scheme. UDOP is pretrained on both large-scale unlabeled document corpora using innovative self-supervised objectives and diverse labeled data. UDOP also learns to generate document images from text and layout modalities via masked image reconstruction. To the best of our knowledge, this is the first time in the field of document AI that one model simultaneously achieves high-quality neural document editing and content customization. Our method sets the state-of-the-art on 9 Document AI tasks, e.g., document understanding and QA, across diverse data domains like finance reports, academic papers, and websites. UDOP ranks first on the leaderboard of the Document Understanding Benchmark (DUE).

We tackle open-world semantic segmentation, which aims at learning to segment arbitrary visual concepts in images, by using only image-text pairs without dense annotations. Existing open-world segmentation methods have shown impressive advances by employing contrastive learning (CL) to learn diverse visual concepts and adapting the learned image-level understanding to the segmentation task. However, these methods based on CL have a discrepancy since it only considers image-text level alignment in training time, while the segmentation task requires region-text level alignment at test time. In this paper, we propose a novel Text-grounded Contrastive Learning (TCL) framework to directly align a text and a region described by the text to address the train-test discrepancy. Our method generates a segmentation mask associated with a given text, extracts grounded image embedding from the masked region, and aligns it with text embedding via TCL. The framework addresses the discrepancy by letting the model learn region-text level alignment instead of image-text level alignment and encourages the model to directly improve the quality of generated segmentation masks. In addition, for a rigorous and fair comparison, we present a unified evaluation protocol with widely used 8 semantic segmentation datasets. TCL achieves state-of-the-art zero-shot segmentation performance with large margins in all datasets. Code is available at https://github.com/kakaobrain/tcl.

We present HOReeNet, which tackles the novel task of manipulating images involving hands, objects, and their interactions. Especially, we are interested in transferring objects of source images to target images and manipulating 3D hand postures to tightly grasp the transferred objects. Furthermore, the manipulation needs to be reflected in the 2D image space. In our reenactment scenario involving hand-object interactions, 3D reconstruction becomes essential as 3D contact reasoning between hands and objects is required to achieve a tight grasp. At the same time, to obtain high-quality 2D images from 3D space, well-designed 3D-to-2D projection and image refinement are required. Our HOReeNet is the first fully differentiable framework proposed for such a task. On hand-object interaction datasets, we compared our HOReeNet to the conventional image translation algorithms and reenactment algorithm. We demonstrated that our approach could achieved the state-of-the-art on the proposed task.

Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities. Previous work addressing privacy issues for language models has mostly focused on data preprocessing and differential privacy methods, both requiring re-training the underlying LM. We propose knowledge unlearning as an alternative method to reduce privacy risks for LMs post hoc. We show that simply performing gradient ascent on target token sequences is effective at forgetting them with little to no degradation of general language modeling performances for larger LMs; it sometimes even substantially improves the underlying LM with just a few iterations. We also find that sequential unlearning is better than trying to unlearn all the data at once and that unlearning is highly dependent on which kind of data (domain) is forgotten. By showing comparisons with a previous data preprocessing method and a decoding method known to mitigate privacy risks for LMs, we show that unlearning can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being much more efficient and robust. We release the code and dataset needed to replicate our results at https://github.com/joeljang/knowledge-unlearning.